System and method for augmented reality visualization of benign paroxysmal position vertigo (bppv) disorder

ABSTRACT

According to one aspect, a system for augmented reality visualization of benign paroxysmal position vertigo (BPPV) disorder. The system includes a camera configured to capture an image sequence, a processing unit configured to generate at least one virtual model of an inner ear, wherein the at least one virtual model of the inner ear comprises an accurate anatomical representation of a real human inner ear; and a display configured to display the at least one virtual model of the inner ear over the image sequence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/892,066 filed on Aug. 27, 2019, which is incorporated by referenceherein in its entirety.

FIELD

The described embodiments relate to a system and method for augmentedreality visualization, and in particular, to a system and method foraugmented reality visualization of benign paroxysmal position vertigo(BPPV) disorder.

INTRODUCTION

The following is not an admission that anything discussed below is partof the prior art or part of the common general knowledge of a personskilled in the art.

Benign paroxysmal position vertigo (BPPV) is a disorder which resultsfrom problems with the vestibular organs of the human inner ear. BPPVmay result, for example, from any injury to the inner ear, illness inthe inner ear, head injury (e.g., head concussions), as well as from oldage. Individuals suffering from BPPV often experience short episodes ofvertigo following specific head movements (e.g., head tilting andturning). The short episodes of vertigo may include, for example, falsesensations of spinning, nausea, loss of balance, and vomiting.

SUMMARY OF THE VARIOUS EMBODIMENTS

According to one broad aspect of the invention, there is disclosed asystem for augmented reality visualization of benign paroxysmal positionvertigo (BPPV) disorder, The system includes a camera configured tocapture an image sequence, a processing unit configured to generate atleast one virtual model of an inner ear, wherein the at least onevirtual model of the inner ear comprises an accurate anatomicalrepresentation of a real human inner ear, and a display configured todisplay the at least one virtual model of the inner ear over the imagesequence.

In some embodiments, the display is configured to display the imagesequence in real-time. In some embodiments the image sequence includesan image of a subject's head. In some embodiments, the at least onevirtual model of the inner ear includes a set of virtual displacedotoconia.

In some embodiments, the at least one virtual model of the inner earcomprises a first virtual model of a right inner ear and a secondvirtual model of a left inner ear. In some embodiments, the processingunit is configured to transmit instructions for the display to displaythe first virtual model on a right side of the image of the subject'shead, and the second virtual model on the left side of the image of thesubject's head.

In some embodiments, the processing unit is further configured tomonitor a movement of the subject's head in the image sequence, and isconfigured to generate a new orientation for at least one virtual modelbased on the monitored movement of the subject's head, and the displayis configured to display the new orientation of at least one virtualmodel.

In some embodiments, the processing unit is further configured tomonitor a movement of the display, and is configured to generate a neworientation for at least one virtual model of the inner ear based on themovement of the display.

In some embodiments, the virtual model of at least one inner ear isrotatable into a new orientation on the display.

In some embodiments, the processing unit is configured to generate a newposition for the virtual displaced otoconia inside at least one virtualmodel of the inner ear, the new position for the displaced otoconia isbased on the new orientation of at least one virtual model of the innerear, and wherein the display is configured to display the new positionof the virtual displaced otoconia.

In some embodiments, the processing unit is configured to code thevirtual displaced otoconia with gravity properties, and is furtherconfigured to generate the new position for the virtual displacedotoconia based on the gravity properties.

In some embodiments, the processing unit is further configured togenerate an augmented set of animated eyes, and the display isconfigured to display the augmented set of animated eyes over the imagesequence.

In some embodiments, the augmented set of animated eyes is configured tofollow a predetermined motion pattern based on a movement of the virtualdisplaced otoconia inside the virtual model of the inner ear.

In some embodiments, the processing unit is configured to monitor amovement of the subject's head in the image sequence based on a visualindicator located on the subject's head.

In some embodiments the processing unit is configured to monitormovement of a subject's head based on reading topographical features ofthe subject's head.

In some embodiments, the camera further comprises an infrared (IR)scanning capability.

In some embodiments, the processing unit is configured to monitor themovement of the subject's head in the image sequence based oninformation generated by the IR scanning capability of the camera.

According to another broad aspect, there is disclosed a method forgenerating an augmented reality model of at least one vestibularlabyrinth. The method includes capturing an image sequence using acamera, using a processing unit, generating at least one virtual modelof an inner ear, wherein the virtual model of the inner ear comprises anaccurate anatomical representation of a real human inner ear, anddisplaying at least one virtual model of the inner ear over the imagesequence.

Other features and advantages of the present application will becomeapparent from the following detailed description taken together with theaccompanying drawings. It should be understood, however, that thedetailed description and the specific examples, while indicatingpreferred embodiments of the application, are given by way ofillustration only, since various changes and modifications within thespirit and scope of the application will become apparent to thoseskilled in the art from this detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

A preferred embodiment of the present invention will now be described indetail with reference to the drawings, in which:

FIG. 1 is a schematic representation of the anatomy of an example humaninner ear;

FIG. 2A is a schematic representation of an example environment foraugmented reality visualization of BPPV, in accordance with someembodiments;

FIG. 2B is a schematic representation of an example environment foraugmented reality visualization of BPPV, in accordance with some otherembodiments;

FIG. 3 is a block diagram of an example user device;

FIG. 4A is a display, of a user device, showing a simulatedvisualization of BPPV, according to some embodiments;

FIG. 4B is a display, of a user device, showing an example augmentedreality visualization of BPPV, according to some other embodiments;

FIG. 4C is a display, of a user device, showing an example augmentedreality visualization of BPPV, according to some further embodiments;and

FIG. 5 is an example process flow for augmented reality visualization ofBPPV.

Further aspects and features of the example embodiments described hereinwill appear from the following description taken together with theaccompanying drawings.

DETAILED DESCRIPTION OF THE EMBODIMENTS

It will be appreciated that for simplicity and clarity of illustration,where considered appropriate, reference numerals may be repeated amongthe figures to indicate corresponding or analogous elements or steps. Inaddition, numerous specific details are set forth in order to provide athorough understanding of the exemplary embodiments described herein.However, it will be understood by those of ordinary skill in the artthat the embodiments described herein may be practiced without thesespecific details, or with other methods, components, materials, etc. Inother instances, well-known methods, procedures and components have notbeen described in detail since these are known to those skilled in theart. Furthermore, it should be noted that this description is notintended to limit the scope of the embodiments described herein, butrather as merely describing one or more exemplary implementations.

Unless the context requires otherwise, throughout the specification andclaims which follow, the word “comprise” and variations thereof, suchas, “comprises” and “comprising” are to be construed in an open,inclusive sense, that is as “including, but not limited to.”

It should be noted that terms of degree such as “substantially”, “about”and “approximately” when used herein mean a reasonable amount ofdeviation of the modified term such that the end result is notsignificantly changed. These terms of degree should be construed asincluding a deviation of the modified term if this deviation would notnegate the meaning of the term it modifies.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structures, orcharacteristics may be combined in any suitable manner in one or moreembodiments.

As used in this specification and the appended claims, the singularforms “a,” “an,” and “the” include plural referents unless the contentclearly dictates otherwise. It should also be noted that the term “or”is generally employed in its broadest sense, that is as meaning “and/or”unless the content clearly dictates otherwise.

The headings and Abstract of the Disclosure provided herein are forconvenience only and do not interpret the scope or meaning of theembodiments.

The terms “coupled” or “coupling” as used herein can have severaldifferent meanings depending in the context in which these terms areused. For example, the terms coupled or coupling may be used to indicatethat an element or device can electrically, optically, or wirelesslysend data to another element or device as well as receive data fromanother element or device.

Similarly, throughout this specification and the appended claims theterm “communicative” as in “communicative pathway,” “communicativecoupling,” and in variants such as “communicatively coupled,” isgenerally used to refer to any engineered arrangement for transferringand/or exchanging information. Exemplary communicative pathways include,but are not limited to, electrically conductive pathways (e.g.,electrically conductive wires, electrically conductive traces), magneticpathways (e.g., magnetic media), optical pathways (e.g., optical fiber),electromagnetically radiative pathways (e.g., radio waves), or anycombination thereof. Exemplary communicative couplings include, but arenot limited to, electrical couplings, magnetic couplings, opticalcouplings, radio couplings, or any combination thereof.

Throughout this specification and the appended claims, infinitive verbforms are often used. Examples include, without limitation: “to detect,”“to provide,” “to transmit,” “to communicate,” “to process,” “to route,”and the like. Unless the specific context requires otherwise, suchinfinitive verb forms are used in an open, inclusive sense, that is as“to, at least, detect,” “to, at least, provide,” “to, at least,transmit,” and so on.

The example embodiments of the systems and methods described herein maybe implemented as a combination of hardware or software. In some cases,the example embodiments described herein may be implemented, at least inpart, by using one or more computer programs, executing on one or moreprogrammable devices comprising at least one processing element, and adata storage element (including volatile memory, non-volatile memory,storage elements, or any combination thereof). These devices may alsohave at least one input device (e.g. a keyboard, mouse, touchscreen, orthe like), and at least one output device (e.g. a display screen, aprinter, a wireless radio, or the like) depending on the nature of thedevice.

It should also be noted that there may be some elements that are used toimplement at least part of one of the embodiments described herein thatmay be implemented via software that is written in a high-level computerprogramming language such as one that employs an object-orientedparadigm. Accordingly, the program code may be written in Java, C++ orany other suitable programming language and may comprise modules orclasses, as is known to those skilled in object-oriented programming.Alternatively, or in addition thereto, some of these elementsimplemented via software may be written in assembly language, machinelanguage or firmware as needed. In either case, the language may be acompiled or interpreted language.

At least some of these software programs may be stored on a storagemedia (e.g. a computer readable medium such as, but not limited to, ROM,EEPROM, magnetic disk, optical disc) or a device that is readable by ageneral or special purpose programmable device. The software programcode, when read by the programmable device, configures the programmabledevice to operate in a new, specific and predefined manner in order toperform at least one of the methods described herein.

The description sets forth various embodiments of the systems, devicesand/or processes via the use of block diagrams, schematics, andexamples. Insofar as such block diagrams, schematics, and examplescontain one or more functions and/or operations, it will be understoodby those skilled in the art that each function and/or operation withinsuch block diagrams, flowcharts, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment, thepresent subject matter may be implemented via Application SpecificIntegrated Circuits (ASICs). However, those skilled in the art willrecognize that the embodiments disclosed herein, in whole or in part,can be equivalently implemented in standard integrated circuits, as oneor more computer programs executed by one or more computers (e.g., asone or more programs running on one or more computer systems), as one ormore programs executed by on one or more controllers (e.g.,microcontrollers) as one or more programs executed by one or moreprocessors (e.g., microprocessors, central processing units, graphicalprocessing units), as firmware, or as virtually any combination thereof,and that designing the circuitry and/or writing the code for thesoftware and or firmware would be well within the skill of one ofordinary skill in the art in light of the teachings of this disclosure.

When logic is implemented as software and stored in memory, logic orinformation can be stored on any processor-readable medium for use by orin connection with any processor-related system or method. In thecontext of this disclosure, a memory is a processor-readable medium thatis an electronic, magnetic, optical, or other physical device or meansthat contains or stores a computer and/or processor program. Logicand/or the information can be embodied in any processor-readable mediumfor use by or in connection with an instruction execution system,apparatus, or device, such as a computer-based system,processor-containing system, or other system that can fetch theinstructions from the instruction execution system, apparatus, or deviceand execute the instructions associated with logic and/or information.

In the context of this specification, a “non-transitorycomputer-readable medium” can be any element that can store the programassociated with logic and/or information for use by or in connectionwith the instruction execution system, apparatus, and/or device. Theprocessor-readable medium can be, for example, but is not limited to, anelectronic, magnetic, optical, electromagnetic, infrared, orsemiconductor system, apparatus or device. More specific examples (anon-exhaustive list) of the computer readable medium would include thefollowing: a portable computer diskette (magnetic, compact flash card,secure digital, or the like), a random access memory (RAM), a read-onlymemory (ROM), an erasable programmable read-only memory (EPROM, EEPROM,or Flash memory), a portable compact disc read-only memory (CDROM),digital tape, and other non-transitory media.

As discussed in the background, benign paroxysmal position vertigo(BPPV) is a disorder which results from problems with the vestibularorgans of the human inner ear. BPPV may result, for example, from anyinjury to the inner ear, illness of the inner ear, head injury (e.g.,head concussions), as well as from old age. Individuals suffering fromBPPV often experience short episodes of vertigo that follow afterspecific head movements (e.g., head tilting and turning). The shortepisodes may include false sensations of spinning, nausea, loss ofbalance, and vomiting.

Referring now briefly to FIG. 1, there is shown a schematicrepresentation of the anatomy of an example human inner ear 100. Theschematic representation of the inner ear 100 is provided herein toallow for a more detailed discussion of BPPV.

The inner ear is generally composed of a spiral-shaped cochlea 102, andthe vestibular labyrinth 104. The cochlea 102 is responsible for sounddetection, while the vestibular labyrinth 104 is responsible fordetecting linear and angular acceleration movement of the head.

The vestibular labyrinth 104 contains a number of sensory organs,including the semi-circular canals 106 and the otolith organs 108. Thesemi-circular canals 106 detect angular acceleration (e.g., rotationalmovement), and include the anterior canal 106 a, the lateral canal 106b, and the posterior canal 106 c. Each canal 106 a-c is fluid-filledwith endolymph fluid, and has an ampulla and a crista lined withmicroscopic sensory hair known as cilia. As the head experiences angularacceleration, the endolymph fluid moves within the canals and causes thecilia hairs to shift and transmit sensory signals to the brain.

In contrast, the otolith organs 108 assist in detecting linearacceleration movement of the head. As shown in FIG. 1, the otolithorgans 108 include the utricle 108 a and the saccule 108 b. Each of theutricle 108 a and the saccule 108 b contains a mass of small biocrystals, known as otoconia, embedded in a gelatinous outer membrane ofthe otolithic organs. During linear acceleration of the head, theotoconia shift and cause sensory hair cells, also embedded in thegelatinous outer layer, to bend and transmit sensory signals back to thebrain.

In general, BPPV occurs when the otoconia become dislodged, ordisplaced, from the otolithic membrane. The displaced otoconiasubsequently migrate—under the force of gravity—into the semi-circularcanals 106, and disrupt the normal functioning of the canals by makingthem sensitive to gravity. For example, small head movements can causethe otoconia 110 to shift inside the canals, under gravitationalinfluence, and in turn, cause the cilia hairs to move, and transmiterroneous (or exaggerated) signals of head rotation to the brain. Thesesignals contradict other sensory perceptions received by the brain(e.g., sight and hearing) and result in the temporary sensation ofvertigo. The sensation of vertigo typically persists until the otoconia110 re-settle inside the canals.

To diagnose and treat BPPV, experienced practitioners apply a sequenceof head and body maneuvers to affected patients. The maneuvers guide theotoconia, using gravitational force, out of the canals and into adesired location within the inner ear. Practitioners often apply thesemaneuvers while observing specific patterns of involuntary eye movementin the patient. These patterns of eye movement, known as nystagmus,assist in identifying the ear which contains the moving otoconia, aswell as the relative position within the ear where the otoconia arelocated. Accordingly, by observing the exhibited patterns of nystagmus,a practitioner may track the motion and position of the otoconia, and inturn, applying corrective maneuvers to guide the otoconia, inside thepatient's ear, back into the otolithic organs.

Necessarily, caution must be exercised during the diagnosis andtreatment of BPPV. In particular, an incorrect sequence of head and bodymaneuvers can aggravate the symptoms of BPPV in a patient. This, inturn, results in increased time and effort for performing remedial andcorrective treatment to the patient.

To this end, it has been appreciated that there are currently fewavailable tools which assist in-experienced practitioners (e.g., medicalpractitioners, or physical therapists) and students to better understandhow to diagnose and treat cases of BPPV. For example, there are fewassistive tools which help visualize movement of displaced otoconiainside the inner ear, as well as to visualize the relationship betweennystagmus and the shifting of displaced otoconia within the ear. Stillfurther, it has been also appreciated that little visual guidance istypically provided to practitioners while treating affected patients.For example, there are no readily available tools which visualize—inreal-time or near real-time—the movement of otoconia inside a patient'sear as the practitioner is applying head and body maneuvers. It isaccordingly expected that by providing practitioners with the ability tovisualize movement of otoconia inside the ear, incidences of erroneoustreatment may be greatly reduced.

In view of the foregoing, embodiments provided herein generally relateto a system and method for augmented reality (AR) visualization ofbenign paroxysmal position vertigo (BPPV) disorder. In particular, thesystem and method provided herein may allow for a better understandingof BPPV in subject patients.

Referring now to FIG. 2A, there is shown a schematic representation ofan example environment 200 a for augmented reality visualization ofBPPV, in accordance with some embodiments.

As shown, the environment 200 a generally includes a user device 202operated by a user 204. User 204 may be, for example, a medicalpractitioner, a physical therapist or a student. User 204 may use userdevice 202 to view an augmented reality visualization of BPPV.

Environment 200 a may also include a subject 208. Subject 208 is, forexample, a patient affected by BPPV, or otherwise, a test subject. TheAR image displayed on user device 202 includes virtual objects (e.g.,virtual inner ear models) projected over an image sequence (e.g., adigital video) of the subject 208, captured in real-time or nearreal-time by a camera in communication with user device 202. In somecases, user 204 and the subject 208 can be the same individual. Forexample, this can occur in cases where the user device 202 is used toself-diagnose, or self-treat BPPV.

In at least some embodiments, one or more visual indicators 210 arelocated, or positioned, over the subject's head 208. For example, asillustrated, the visual indicators 210 are attached to a head band 212positioned around the subject's forehead. As explained herein, thevisual indicators 210 can be used to detect the location of thesubject's head in an image sequence captured by a user device camera. Inparticular, the visual indicators 210 can synchronize virtual objectsdisplayed on the user device 202 with movement (e.g., rotation) of thesubject's head in a captured image sequence. In some cases, a cameraequipped with IR scanning functionality or otherwise, an applicationconfigured for image analysis, may also be used to detect the positionof the subject's head in the captured image sequence. In these cases,the use of visual indicators 210 may not be necessary. In someembodiments, the user device 202 can also be equipped with a LiDARsensor. The LiDAR sensor can be used to detect the visual indicators210. In other cases, the LiDAR sensor can also scan the environment, andgenerate LiDAR sensor data which can be analyzed to identify objects orfeatures corresponding to a patient's head. For example, the LiDARsensor data can be used to identify recognizable landmarks on thepatient's head (e.g., eyes, nose, mouth, ears, chin, and jaw), which, inturn, can help identify the patient's head position and orientation.

Referring now to FIG. 2B, there is shown a schematic representation ofan example environment 200 b for augmented reality visualization ofBPPV, in accordance with some other embodiments.

As shown, the user device 202 may also be a head-mounted device thatstraps around the user's head. For example, as illustrated, the userdevice 202 is located inside a hands-free set which straps around theuser's head using a head strap 206. In this configuration, the user'shands are freed for occupation with other functions. For example, theuser 204 (e.g., a medical practitioner or physical therapist) can occupytheir hands with applying a sequence of head and body maneuvers to thesubject 208 while concurrently viewing the AR environment on user devicedisplay. The head-mounted device also provides the user 204 with a moreimmersive AR experience.

It will be appreciated that while the user device 204 has beenillustrated herein as being either a hand-held device or a head-mounteddevice, the user device 202 can also be accessible to user 204 in anyother manner. For example, the user device 202 can be mounted onto astationary mount unit.

It will also be appreciated that the environments 200 a and 200 b maynot necessarily include subject 208. For example, in some embodiments,the user device 202 can simply project virtual objects over anyenvironment which surrounds the user 204. In still other cases, thevirtual objects can be projected over an artificially generatedenvironment (e.g., a simulated environment), generated by an ARapplication operating on the user device 202.

Referring now to FIG. 3, there is shown a simplified block diagram 300of an example user device 202.

User device 202 may be, for example, a mobile user device (e.g., asshown in FIG. 2A) but may also include any suitable computing devicecapable of executing an application. For instance, user device 202 mayalso be a desktop or laptop computer, as well as a smartphones, tabletcomputers, as well as a wide variety of “smart” devices capable of datacommunication. Increasingly, this encompasses a wide variety of devicesas more devices become networked through the “Internet of Things”.

As shown, user device 202 may generally include a processor 302, amemory 304, a camera 306, a display 308, a user interface 310, sensors312, an input/output (I/O) interface 314, a communication interface 316and an augmented reality (AR) program 318. In various cases, user device202 may also include speakers 320.

Processor 302 is a computer processor, such as a general purposemicroprocessor. In some other cases, processor 302 may be a fieldprogrammable gate array, application specific integrated circuit,microcontroller, or other suitable computer processor.

Processor 302 is coupled, via a computer data bus, to memory 304. Memory304 may include both volatile and non-volatile memory. Non-volatilememory stores computer programs consisting of computer-executableinstructions, which may be loaded into the volatile memory for executionby processor 302 as needed. It will be understood by those of skill inthe art that references herein to user device 202 as carrying out afunction or acting in a particular way imply that processor 302 isexecuting instructions (e.g., a software program) stored in memory 304and possibly transmitting or receiving inputs and outputs via one ormore interfaces. Memory 304 may also store data input to, or outputfrom, processor 302 in the course of executing the computer-executableinstructions. In various cases, augmented reality (AR) program 318 mayalso be stored on memory 304.

Camera 306 is generally a digital camera, which captures digital images,or a digital video camera, which captures consecutive digital imageframes. For example, as shown in FIGS. 2A and 2B, camera 306 may beconfigured to capture a real-time, or near real-time, digital video of asubject 208, or otherwise capture a digital video of a user'ssurrounding environment. In at least some cases, camera 306 may beintegrated in user device 202, but may also be a separate device that isotherwise in communication with the user device 202. For example, insome cases, the camera 306 may be a separate device in wired or wirelesscommunication with the user device 202 (e.g., via communicationinterface 316).

In at least some embodiments, camera 306 may be configured with infrared(IR) scanning functionality, and may include infrared photodetectorsconfigured to detect heat radiation. In at least some cases, asexplained in further detail herein, the IR information generated bycamera 306 may be used to identify the location of specific objectslocated in a user's surrounding environment. For example, the IRinformation may be used to identify the location of a subject's head 208in a captured image sequence. In other cases, the IR camera may beprovided as a separate device in communication with user device 202.

Display 308 may be any suitable display for outputting information anddata as needed by various computer programs. For example, display 308may be a screen integrated in user device 202, or otherwise incommunication with user device 202. In various cases, the display 308may be configured to display an augmented reality or virtual realityenvironment for visualizing BPPV. For example, in various embodimentsexplained herein, display 308 may receive and display a video feedcaptured by camera 306 in real-time or near real-time of a user'ssurrounding environment. The display 308 may then display renderedvirtual objects projected over the sequence of captured image frames.For instance, as explained herein, display 308 may display virtualmodels of example human inner ears projected over images frames capturedby camera 306. In other cases, display 308 may also display a virtual(e.g., simulated) environment, and may project the virtual objectsinside the virtual environment.

In at least some embodiments, display 308 may be a touch-screen display.For example, display 308 may be a resistive or capacitive touchscreenwhich is configured to detect touch force applied by a user 204 of userdevice 202. In various cases, this may allow a user 204 to use thetouchscreen to interact with virtual objects displayed on display 308.In other cases, a separate input interface (e.g., keyboard and mouse)may be provided for receiving user inputs.

In some embodiments, display 308 may also display a graphical userinterface (GUI). For example, as explained herein, the GUI may provide auser friendly environment for viewing and interacting with virtualobjects and images displayed on display 308.

User interface 310 may be one or more devices that allow a user, oroperator, to interact with the user device 202. For example, the userinterface 310 may have a keyboard or other input device that allows auser to input instructions into the user device 202. For example, invarious cases, the user may input instructions for camera 306 to capturea digital video of a user's surrounding environment. In other cases, theuser interface 310 may allow the user to perform functions previouslydescribed with respect to the touchscreen display 308.

User device 202 may also include one or more sensors 312, which may beconfigured to detect motion of the user device 202. Sensors 312 mayinclude, for example, an inertial measurement unit (IMU), which mayinclude at least a gyroscope and one or more accelerometers. The IMU maybe detect movement and rotation of the user device 202. For example, auser 204 may rotate the user device 202 to rotate the camera view. Byrotating the camera view, the user may view, on display 308, differentperspectives of a virtual model of an example inner ear which isprojected on the user's surrounding environment (or a virtualenvironment). Accordingly, the IMU may detect rotation (and movement) ofuser device 202, and may transmit the rotational (and positional)information to AR program 318. The AR program 318 may receive therotational (and positional) information from the IMU, and may update thedisplayed virtual model based on the received rotational (andpositional) information. Accordingly, a user of user device 202 mayperceive the virtual model from different perspectives by rotating (andmoving) the user device. In other cases, sensors 312 can also include aLiDAR sensor. As explained herein, the LiDAR sensor can be used toidentify visual indicators 210 attached to a patient's head. In othercases, generated LiDAR sensor data can also be used to identifyrecognizable features indicating the location and position of apatient's head.

Input/output (I/O) interface 314 may allow for coupling of other devicesto the user device 204. For example, in some cases, the camera 306and/or display 308 may not be integrated into the user device 202, andmay be coupled to the user device 202 via I/O interface 314. In othercases, an integrated camera may not be configured with IR scanningfunctionality, and an IR camera may also be coupled to user device 202via I/O interface 314.

Communication interface 316 may be one or more data network interface,such as an IEEE 802.3 or IEEE 802.11 interface, for communication over anetwork with other components.

Augmented reality (AR) program 318 may be, for example, a stand-aloneapplication located on a mobile user device 202, or a program located ona desktop computer. In other cases, AR program 318 may be a plug-in oran extension for a web-browser interface located on user device 202.

AR program 318 may be configured to generate (e.g., render) and transmitvirtual objects for display on display 308. The virtual objects may berendered, for example, using data retrieved from memory 304 (e.g., datastored on memory 304), or otherwise, from data received from an externalserver in communication with user device 202 (e.g., via communicationinterface 316).

In various embodiments, the AR program 318 may operate in conjunctionwith the camera 306. For example, the camera 306 may capture a sequenceof images (e.g., a digital video) in real-time—or near real-time—of auser's surrounding environment. The AR program 318 may receive the imagesequence, and may render one or more virtual objects over the capturedimage sequence to generate an augmented reality environment. The imagesgenerated by the AR program 318 may then be transmitted for display onthe display 308 in real-time, or near real-time. AR program 318 may alsorender (e.g., project) virtual objects over an artificial or simulatedenvironment, which may also be generated by the AR program 318.

In some cases, virtual objects rendered by AR program 318 may includevirtual models of example human inner ears. For example, AR program 318may render a two-dimensional (2D), or a three-dimensional (3D) virtualinner ear model (e.g., an anatomical 3D model of the inner ear, as shownin FIG. 1). The rendered virtual inner ear model may allow a user 204 tobetter visualize and understand the anatomy of the human ear.

Referring now briefly to FIGS. 4A—4C, which show various exampledisplays 308, of a user device 202. As shown in each of these figures,the AR program 318 may generate and render one or more inner ear models(e.g., a right inner ear model 404 a and a left inner ear model 404 b),for display on the display 308.

AR program 318 may also adjust features of the rendered virtual earmodels. The features may be adjusted in response to user inputs (e.g.,using a touchscreen display 308 or user interface 310), or otherwiseautomatically adjusted in response to certain user or patient actions.

For example, AR program 318 may adjust the number of ear modelsdisplayed on display 308. In other words, AR program 318 may render botha left inner ear model and a right inner model to allow a user to viewdifferent ear models. AR program 318 can also render only a desiredportion of an inner ear model. Still further, AR program 318 can alsorender a plurality of left and/or right inner ear models. In some cases,this can allow a user to view different variations of inner ear modelssimultaneously. For example, a user may wish to view multiple ear modelshaving virtual otoconia located in different positions as explainedherein, or different ear models rotated in different positions.

AR program 318 may also adjust the transparency of virtual ear models.For example, virtual ear models may be made more transparent, orotherwise, more opaque. The virtual models may be made more transparent,for example, to allow viewing of the inner anatomy of the virtual ears.In some cases, AR program 318 may also generate different sectionalviews of the virtual models. For instance, AR program 318 may generatedifferent cross-sectional views that allow a user to observe inside theear. This feature may be provided in addition to, or in alternative to,adjusting the model transparency. The transparency and cross-sectionalviews of the ear models can be adjusted, for example, by a user usingthe user interface 310.

As explained herein, AR program 318 may also adjust the viewingperspective, or angular view, of a 3D virtual ear model. For example, ARprogram 318 can allow rotation of a virtual model in response to areceived user input (e.g., the user may rotate the virtual ear modelusing a GUI interface on display 308). For example, FIG. 4A shows a GUIwhich includes a graphic element 410 a for zooming-in or zooming-out ofthe virtual models 404, as well as a graphic element 410 b for rotatingone or more of the virtual models 404.

AR program 318 may also rotate the view of the ear models based oninformation received from sensors 312 (e.g., an NU). For example, insome cases, the virtual models may be projected on an image sequence(e.g., digital video), captured by camera 306—in real-time or nearreal-time—and responsive to the user rotating the user device 202 (orthe camera view of camera 306), AR program 318 can correspondinglyadjust the displayed perspective view of the virtual model. For example,a user may move (e.g., rotate) the user device 202 around the subject'shead, and the displayed perspective view of the ear models cancorrespondingly change in real-time, or near real-time, with movement ofthe user device 202 to show front, side and rear views. In still otherembodiments explained herein, the AR program 318 may synchronizerotation of the virtual ear models with detected rotation of an actualsubject's head.

Accordingly, it will be appreciated that the AR program 318 may vary awide array of features relating to the virtual ear models in order toenhance user experience of the augmented reality or virtual environment.

In various cases, AR program 318 can also simulate BPPV. This can bedone by displaying a set of virtual displaced otoconia inside of atleast one rendered ear model. For example, the AR program 318 maygenerate a virtual ear model having virtualized otoconia located, forexample, within a semi-circular canal. For examples, FIGS. 4A-4C showsexample virtual otoconia 406 displayed inside of an ear model 404 a.

The virtual otoconia can be rendered by the AR program 318 in differentlocations within the virtual ears, or in different ears (e.g., left orright). For example, AR program 318 may change the location of thevirtual otoconia in response to receiving a user input (e.g., a user maychange the location of the virtual otoconia using an input device). Thelocation of the virtual otoconia can also be automatically adjusted bythe AR program 318 in response to user or patient actions, as explainedherein. Various other features of the virtualized otoconia can also beadjusted (e.g., size and shape). The AR program 318 can also toggledisplay of the virtual otoconia, e.g., as desired by the user.

To simulate BPPV, the AR program 318 may further visualize shifting(e.g., movement) of the displaced otoconia within the virtual ears. Forexample, AR program 318 may render the otoconia moving within thevirtual inner ear, under the simulated effect of gravity (e.g., thevirtual otoconia maybe coded with gravity properties). For example,virtual otoconia may move within the virtual ear in response to movementand rotation of the virtual ear. FIGS. 4B and 4C, for example,illustrate octonia 406 shifting within the ear model 404 a in responseto rotation of the ear model, and under the simulated effect of gravity.

When visualizing movement of the octoconia, AR program 318 may accountfor various factors which may affect the movement of the otoconia insidethe ear. For example, the AR program 318 may account for fluid frictionin the canals (e.g., viscous friction) which otherwise slow down thespeed of movement of the otoconia. The AR program 318 may also accountfor friction between the otoconia and the inner wall of the canals,which may also effect the speed or direction of otoconia movement.

In some cases, AR program 318 may shift the otoconia inside of thevirtual inner ear model in response to rotation of the inner ear models(e.g., by the user). Accordingly, this feature may allow a user 204(e.g., a medical or physical therapy students) to observe how movementof the inner ear (e.g., resulting from head movement) influencesshifting of the otoconia inside the ear. In some cases, a user 204 maymove (e.g., rotate) the virtual models in different directions (e.g.,using an input device), and may observe, on display 308, how eachmovement affects the shifting of the virtual otoconia. In at least somecases, a user 204 may also practice moving (e.g., rotating) the virtualear models in order to guide the virtual otoconia within the ear to adesired location. For instance, a user may practice moving the ear modelto guide virtual otoconia out of the semi-circular canals, and into theotolith organs. Accordingly, the virtual ear models may act as anassistive learning tool for students or in-experienced practitioners tobetter understand BPPV.

Referring now briefly to FIG. 4A, there is shown a display 308, of userdevice 202, showing a simulated visualization of BPPV, according to someembodiments.

As shown, in various cases, the AR program 318 may render the virtualear models 404 in conjunction with an “avatar head” 402. The avatar head402 may simulate, for example, a hypothetical patient affected by BPPVdisorder.

The avatar head 402 may be displayed on display 308 in-between the pairof virtual ears 404 (e.g., a right virtual ear model 404 a and a leftvirtual ear model 404 b, or at least one ear model). The ear models 404may be synchronized with movement of the avatar head 402. In theillustrated example, the avatar head 402 is rotated to the left by auser 204 of user device 202 (e.g., using a touchscreen display 308), andthe virtual ear models 404 are also correspondingly rotated to the left.The right virtual ear model 404 a is also illustrated with the set ofvirtual otoconia 406. As shown, the virtual otoconia 406 shifts withinthe virtual ear model 404 a, under the simulated influence of gravity,as a result of the tilting of the avatar head 402. The avatar head 402is also illustrated with a set of animated eyes 408 which are configuredto simulate the effect of nystagmus, resulting from shifting of thevirtual otoconia 406 inside the virtual ear 404 a.

In various cases, the use of an avatar head 402 may allow users tounderstand how different head positions result in different movement ofthe virtual otoconia 406 inside the inner ear 404. The avatar head 402may also enable users to learn to apply the correct sequence of head andbody maneuvers for treating hypothetical cases of BPPV. For example, auser may move the avatar head 402 to guide the virtual otoconia 406 outof the virtual canals, and back into the otolith organs. Accordingly,the avatar head 402 may provide students and in-experiencedpractitioners, alike, with a simulated environment for learning to treatBPPV. As explained in further detail herein, in at least some cases, ARprogram 318 may also generate visual (e.g., textual or graphic),auditory or tactile instructions which may guide a user in treating thehypothetical case of BPPV using the avatar head 402.

Still referring to FIG. 4A, AR program 318 may also render a set ofvirtual animated eyes 408 on display 308. The animated eyes 408 may be,for example, displayed in conjunction with avatar head 402, orotherwise, separately (e.g., independently) from the avatar head 402(e.g., the avatar head 402 may not be displayed, and only the eyes 408may be displayed). The animated eyes 408 may, for example, replicatepatterns of nystagmus. For example, the animated eyes 408 may displaydifferent eye movement patterns resulting from movement of the virtualotoconia 406 inside the virtual ear models 404. For instance, a user maymove the virtual ear models 404 (or the avatar head 402) in order toshift the virtual otoconia 406, and may observe the resulting pattern ofnystagmus on the animated eyes 408. Accordingly, this may allow users(e.g., students or in-experienced practitioners) to gain a betterunderstanding of the relationship between patterns of nystagmus andmovement of the otoconia within the inner ear.

The combination of the animated eyes 408, avatar head 402 and thevirtual ear models 404 may, in various cases, serve as a testing toolfor evaluating users ability to diagnose BPPV. For example, the ARprogram 318 may display the avatar head 402 in a pre-defined position(e.g., in a tilted position), and the animated eyes 408 may be made tofollow a pre-defined pattern of nystagmus based on the avatar head'sposition. Further, the position of the virtual otoconia 406 may behidden from view to the user. Based on the displayed avatar headposition and the pattern of nystagmus, the user may be asked to predicta location for the virtual otoconia 406 inside the virtual ear model(e.g., using an input device). The AR program 318 may then display(e.g., automatically, or in response to a user input) the correctposition for the virtual otoconia 406, thereby allowing the user toevaluate their ability to correctly analyze head positions and nystagmuspatterns to correctly identify the location of the virtual otoconiawithin the inner ear.

Using the avatar head 402, AR program 318 can also allow users to testtheir ability to treat BPPV (i.e., in addition to diagnosing BPPV). Forexample, AR program 318 may display the avatar head 402, the animatedeyes 408, and one or more virtual ear models 404. The AR program 318 maythen momentarily display an initial position for the virtual otoconia406 within at least one virtual ear model 404. The AR program 318 maythen hide the display of the otoconia 406 within the virtual ear model404 (or otherwise, the user may select to hide the otoconia fromdisplay). The user may then be asked to apply a sequence of maneuvers tothe avatar head 402 (or the virtual ear models 404) to treat thesimulated case of BPPV. After completing the sequence of movements, theAR program 318 may then display the final location of the virtualotoconia 406 resulting from the sequence of movements applied to theavatar head 402. Accordingly, a user may observe the final position ofthe otoconia to determine whether they have applied the correct sequenceof maneuvers which properly guide the virtual otoconia 406 back into theotolith organs.

In view of the foregoing, it will be appreciated that the combination ofthe animated eyes 408, avatar head 402 and the virtual ear models 404may provide for a beneficial tool for understanding BPPV, as well as forevaluating the ability of a user to diagnose and treat BPPV.

Referring now briefly to FIGS. 4B and 4C, there are illustrated variousexample displays 308, of user device 202, showing various simulatedvisualizations of BPPV, according to some other embodiments.

As shown, AR program 318 may also be configured to position virtual earmodels 404 a, 404 b over captured images of an actual subject patient'shead 208. In particular, as explained herein, this feature may allow theAR program 318 to assist practitioners in real-time diagnosis andtreatment of actual subjects suffering from BPPV. For example, camera306—of user device 202—may capture an image sequence of subject 208(e.g., a digital video)—in real-time or near real-time. AR program 318may receive the image sequence, and relatively position virtual modelsof a right inner ear 404 a and a left inner ear 404 b over the image ofthe subject's head.

In order to synchronize the rendered virtual ear models 404 with thesubject's head 208 (e.g., for synchronized movement or rotation), visualindicators 210 may be positioned on the subject 208. For example, visualindicators 210 may be located on a head band 212 placed around thesubject's forehead. In other cases, the visual indicators 210 may belocated in any other region of the subject's head or body, and may beattached to the subject in any other suitable manner. The visualindicators 210 may be, for example, a scannable pattern such as atwo-dimensional barcode, a quick response (QR) code, or a visual text ordesign. The AR program 318 may receive and analyze an image, or imagesequence (e.g., digital video) from the camera in order to identify anddetermine the location of the visual indicators 210. In other cases, thevisual indicators 210 may be automatically detected by one or moresensors 312 (e.g., LiDAR sensors detecting unique LiDAR-detectable QRcodes). Based on the identification of the visual indicators 210, the ARprogram 318 may locate the user's head in the image, and may relativelyposition the virtual ear models to the subject's head on display 308. Invarious cases, the visual indicators 210 may be positioned to allow theAR program 318 to identify the Reid's line of the patient 208, and toposition the virtual ear models 404 a, 404 b in respect of the Reidline. For example, the visual indicators 210 may be positioned along aplane of, or slightly offset by a pre-determined distance from the planedefining the Reid's line. The AR program 318 may then position thevirtual ear models at the same plane as visual indicators, are at apre-defined offset distance known to the AR program 318.

In other cases, in addition to or in the alternative of using visualindicators, the AR program 318 may work in conjunction with other imageprocessing programs located on the user device 202 to analyze an imageor digital environment representation to identify—in the capturedimage—pre-determined shapes corresponding to a patient's head, as wellas identifiable anatomical features (e.g., eyes, nose, ear, chin, etc.)which indicate head orientation and position. In some cases, the ARprogram 318 (or other suitable programs) may be pre-programmed (e.g.,pre-trained) to identify known features corresponding to a patient'shead, as well as to various known anatomical features.

The AR program 318 may also identify the location and orientation of auser's head in an image based on IR data received from an IR scanner.For example, the camera 306 may be configured with IR scanningfunctionality, or in other cases, an IR camera may be coupled to I/Ointerface 314 of user device 202. The AR program 318 may receive thermalimaging data from the IR scanner, and may use the thermal imaging datato locate the position of the subject's head in the image sequence, aswell as identifiable anatomical features (e.g., eyes, nose, ear, chin,etc.) which indicate head orientation and position. In still otherembodiments, the location and orientation of the user's head can bedetermined based on data received from other sensors 312. For example,data generated by a LiDAR sensor (or other time-of-flight sensorsincluded in sensors 312) can be analyzed by the AR program 318 to detectrecognizable shapes corresponding to a patient's head, as well asdistinctive anatomical features (e.g., eyes, nose, ear, chin, etc.),which can be used to determine head position and orientation.

As shown in FIG. 4C, the AR program 318 may also synchronize theposition and rotation of the virtual ear models 404 a, 404 b with thedetected movement of the subject's head 208 (e.g., based on detectmovement of visual indicators 210). For example, the AR program 318 mayanalyze image frames to detect movement of the subject's head (e.g., viadetecting movement of the visual indicators 210 in the images, orthrough image analysis to determine movement of a head shape/anatomicalfeatures in the images), and may re-position (e.g., move) and rotate thevirtual ear models 404 a, 404 b in synchronization with the subject'shead on display 308. In other cases, head movement can also be detectedbased on IR or LiDAR sensor data. In particular, as shown in FIG. 4C,subject 208 has rotated (e.g., tilted) their head to the left. Forinstance, the subject may have tilted their head in response toinstructions received from a medical practitioner or physical therapist(e.g., user 204). The head rotation is detected by AR program 318, andthe AR program 318 may accordingly re-position and rotate the virtualear models 404 to synchronize the orientation of the virtual ear modelswith the detected head rotation.

In some embodiments, as the virtual models 404 a, 404 b are re-oriented(e.g., tilted) in response to movement of the subject's head, the ARprogram 318 can also display shifting of the virtual otoconia 406 insidethe virtual ear models. This feature may provide a user (e.g., a medicalpractitioner or physical therapist), for example, with a visual guidefor treating the subject 208 diagnosed with BPPV.

In some cases, the AR program 318 may allow a medical practitioner toapply a sequence of maneuvers to an actual subject's head, all thewhile, tracking shifting of the virtual otoconia 406 inside the virtualear models 404 based on display 308. Accordingly, observing the shiftingof the virtual otoconia 406 may assist the practitioner in applying thecorrect sequence of maneuvers which move the otoconia out of the canalsand back into the otolith organs. In this manner, the augmented realityenvironment may provide an assistive tool for more effective treatmentof BPPV.

As shown in FIG. 4C, AR program 318 can also generate a set of animatedeyes 408, to be displayed on display 308, in conjunction with the imagesequence of the subject's head. The set of animated eyes 408 maydisplay, for example, patterns of nystagmus resulting from movement ofvirtual otoconia 406 inside the synchronized virtual ear models. A user(e.g., a medical practitioner or physical therapist), of user device202, may observe the animated pattern of nystagmus, and compare theanimated pattern of nystagmus to the actual subject's nystagmus (e.g.,412 in FIG. 4C).

Where the animated and actual pattern of nystagmus are not identical,the virtual ear models 404 a, 404 b may be deemed inaccurate. This canoccur, for example, where the virtual otoconia 406—inside the virtualear models 404 a, 404 b—incorrectly reflect the actual position of theotoconia inside the actual patient subject's inner ear. To correct forthis problem, the user 202—of user device 204—can adjust the virtualmodel by re-positioning the virtual otoconia. Accordingly, the animatedeyes 408 may be used for verifying the accuracy of the virtual earmodels 404 a, 404 b.

In other cases, the AR program 318 may analyze an image sequence todetermine the actual subject's pattern of nystagmus. For example, the ARprogram 318 may include an image analysis feature which analyzesreceived images to determine movement of the subject's eyes. In othercases, the AR program 318 may cooperate with a separate applicationwhich analyzes images to identify eye movement in an image sequence of asubject. Based on the identified pattern of eye movement, the AR program318 may update the position of the virtual otoconia 406 inside thevirtual ear to reflect the correct position of the otoconia inside thesubject's ear. Accordingly, the AR program 318 may be automaticallyconfigured to correct the virtual ear model with no input from a user.

Referring now back to FIG. 3, user device 202 may also include speakers320. Speakers 320 can be used to provide auditory instructions to a user204, of user device 202, for treating a case of BPPV. For example, asshown in FIG. 4A, the speakers 320 may provide instructions to a user(e.g., a medical student) for treating a simulated case of BPPV usingthe avatar head. Accordingly, the user may follow the auditory guidanceto perform the correct sequence of maneuvers to the avatar head in orderto guide the virtual otoconia 406 to a desired location. In other cases,the auditory instructions may assist a user (e.g., a medial practitioneror physical therapists) in treating an actual patient 208, as shown inFIGS. 4B and 4C. For instance, the instructions may guide thepractitioner in applying the correct sequence of movements to subject208 in order to guide the virtual otoconia to the otolith organs, andaccordingly, treat the subject patient. In various cases, the auditoryinstructions may be generated by the AR program 318.

In other cases, as explained previously, the instructions may also bevisual (e.g., textual or graphical instructions on display 308), ortactile.

Referring now to FIG. 5, there is shown an example process flow for amethod 500 for augmented reality visualization of BPPV. The method 500may be carried out, for example, using the processor 302 of the userdevice 202.

At 502, the camera 306 may be used to capture an image sequence of asubject 208 (e.g., a digital video).

At 504, the AR program 318 may receive the image sequence in real-timeor near real-time, and may locate and determine the position of thesubject's head 208 in the image sequence. For example, the AR program318 may identify visual indicators 210 located on the subject's headwhich identify the position of the subject's head in the image sequence.In other cases, the AR program 318 may receive information from an IRscanning device (e.g., integrated into camera 306, or otherwiseseparately provided and in communication with user device 202). In stillother cases, the AR program 318 may be configured—alone and/or inconjunction with a separate image analysis application—to analyze thereceived images to identify an image object that corresponds to thesubject's head, as well as other image objects that correspond to otheranatomical features of the head. In still yet other cases, the ARprogram 318 may locate the position of the subject's head, as well asvarious anatomical features, based on other received sensor data (e.g.,LiDAR sensor data).

At 506, the AR program 318 may project (e.g., render) one or morevirtual inner ear models, relatively positioned to the located image ofthe subject's head, in the image sequence in real-time, or nearreal-time. For example, the AR program 318 may render a right virtualinner ear model to the right of the subject's head, and a left virtualinner ear model to the left of the subject's head. In some cases, atleast one or both of the virtual ear models may include a virtual set ofotoconia. The AR program 318 may also render a set of animated eyes overthe image of the subject's head.

At 508, the AR program 318 may detect rotation or head movement of theactual subject's head in the image sequence. For example, the headrotation or movement may be detected by tracking the movement of thevisual indicators 210 within the received image sequence. For example,this can occur using an image analysis program which is configured todetect a recognizable and known pattern associated with the visualindicators 210. In other cases, the visual indicators 210 can be trackedbased on other sensor data (e.g., LiDAR sensor data, or othertime-of-flight sensor data). In still other cases, the head rotation ormovement may be detected based on information received from an IRscanning device, or otherwise from LiDAR sensor data (or othertime-of-flight sensor data).

At 510, the AR program 318 may adjust the virtual ear models tosynchronize with the subject's head movement or rotation. For example,the virtual ear models may be rotated in synchronization with the actualsubject's head rotation. In various cases, the virtual otoliths may alsobe shifted within the virtual ear model, under the simulated effect ofgravity, in response to the rotation of the virtual ear models. In stillother cases, the set of animated eyes may display a predicted (orexpected) pattern of nystagmus, based on the shifting of the virtualotoconia inside the virtual ear model.

The present invention has been described here by way of example only,while numerous specific details are set forth herein in order to providea thorough understanding of the exemplary embodiments described herein.However, it will be understood by those of ordinary skill in the artthese embodiments may, in some cases, be practiced without thesespecific details. In other instances, well-known methods, procedures andcomponents have not been described in detail so as not to obscure thedescription of the embodiments. Various modifications and variations maybe made to these exemplary embodiments without departing from the spiritand scope of the invention, which is limited only by the appendedclaims.

1. A system for augmented reality visualization of benign paroxysmalposition vertigo (BPPV) disorder, the system comprising: a cameraconfigured to capture an image sequence; a processing unit configured togenerate at least one virtual model of an inner ear, wherein the atleast one virtual model of the inner ear comprises an accurateanatomical representation of a real human inner ear; and a displayconfigured to display the at least one virtual model of the inner earover the image sequence.
 2. The system of claim1, wherein the imagesequence includes an image of a subject's head.
 3. The system of claim1, wherein the at least one virtual model of the inner ear includes aset of virtual displaced otoconia coded with gravity properties.
 4. Thesystem of claim 3, wherein the at least one virtual model of the innerear comprises a first virtual model of a right inner ear and a secondvirtual model of a left inner ear, and the processing unit is configuredto transmit instructions for the display to display the first virtualmodel on a right side of the image of the subject's head, and the secondvirtual model on the left side of the image of the subject's head. 5.The system of any one of claim 4, wherein the processing unit is furtherconfigured to monitor a movement of the subject's head in the imagesequence, and is configured to generate a new orientation for the atleast one virtual model based on the monitored movement of the subject'shead, and the display is configured to display the new orientation ofthe at least one virtual model.
 6. The system of claim 5, wherein theprocessing unit is configured to generate a new position for the virtualdisplaced otoconia inside the at least one virtual model of the innerear, the new position for the displaced otoconia is based on the neworientation of the at least one virtual model of the inner ear, andwherein the display is configured to display the new position of thevirtual displaced otoconia based on the coded gravity properties.
 7. Thesystem of claim 6, wherein the processing unit is further configured togenerate an augmented set of animated eyes, and the display isconfigured to display the augmented set of animated eyes over the imagesequence, the augmented set of animated being configured to follow apredetermined motion pattern based on a movement of the virtualdisplaced otoconia inside the at least one virtual model of the innerear.
 8. The system of claim 5, wherein the processing unit is configuredto monitor a movement of the subject's head in the image sequence basedon a visual indicator located on the image of the subject's head.
 9. Thesystem of claim 8, wherein the camera further comprises an infrared (IR)scanning capability, and the processing unit is configured to monitorthe movement of the subject's head in the image sequence based oninformation generated by the IR scanning capability of the camera. 10.The system of claim 5, wherein the processing unit is configured tomonitor a movement of the subject's head based on data generated by atime of flight sensor.
 11. A method for generating an augmented realitymodel of at least one vestibular labyrinth, the method comprising:capturing an image sequence using a camera; using a processing unit,generating at least one virtual model of an inner ear, wherein thevirtual model of the inner ear comprises an accurate anatomicalrepresentation of a real human inner ear; and displaying at least onevirtual model of the inner ear over the image sequence.
 12. The methodof claim 11, wherein the image sequence includes an image of a subject'shead.
 13. The method of claim 12, wherein the at least one virtual modelof the inner ear includes a set of virtual displaced otoconia which arecoded with gravity properties.
 14. The method of claim 13, wherein theat least one virtual model of the inner ear comprises a first virtualmodel of a right inner ear and a second virtual model of a left innerear, and the method further comprises transmitting instructions for thedisplay to display the first virtual model on a right side of the imageof the subject's head, and the second virtual model on the left side ofthe image of the subject's head.
 15. The method of claim 14, furthercomprising monitoring a movement of the subject's head in the imagesequence, and generating a new orientation for the at least one virtualmodel based on the monitored movement of the subject's head, anddisplaying the new orientation of the at least one virtual model. 16.The method of claim 15, further comprising generating a new position forthe virtual displaced otoconia inside the at least one virtual model ofthe inner ear, the new position for the displaced otoconia is based onthe new orientation of the at least one virtual model of the inner ear,and displaying the new position of the virtual displaced otoconia isbased on the coded gravity properties.
 17. The method of claim 16,further comprising generating an augmented set of animated eyes, anddisplaying the augmented set of animated eyes over the image sequence,the augmented set of animated eyes being configured to follow apredetermined motion pattern based on a movement of the virtualdisplaced otoconia inside the at least one virtual model of the innerear.
 18. The method of claim 15, further comprising monitoring amovement of the subject's head in the image sequence based on a visualindicator located on the image of the subject's head.
 19. The method ofclaim 18, wherein the camera further comprises an infrared (IR) scanningcapability, and the method further comprises monitoring movement of thesubject's head in the image sequence based on information generated bythe IR scanning capability of the camera.
 20. The method of claim 15,further comprising monitoring a movement of the subject's head based ongenerated by a time-of-flight sensor.